1. Field of the Invention
The present invention relates generally to computer display systems, and more particularly to voice controlled cursor positioning for computer display systems.
2. Description of the Related Art
The use of cursors in a multi-display environment can be problematic if the cursor is either “lost” or a long distance from where it is desired. This can be especially true, for example, in an aircraft cockpit environment with several displays utilizing cursors to access or manipulate data on one of several screens. The cursor is often blocked by, for example, flight critical information. Furthermore, because of the motion of the aircraft the cursor may bounce around. Manipulation of the cursors in aircraft is typically provided by a trackball or a joystick. It would be desirable for the user to have a means to instantly move the cursor to a region of interest.
There are numerous patents that have issued that have involved voice recognition in conjunction with cursor positioning in computer displays. These include the following:
U.S. Pat. No. 5,920,841, entitled “Speech Supported Navigation of a Pointer in a Graphical User Interface,” discloses a method and an apparatus for speech controlled navigation of a pointer in a graphical user interface. The method and apparatus therefore provides a space of discrete position states (quantization) for the pointer which can be navigated only via those discrete positions by means of speech command input. The granularity of the discrete position states can be adapted to the respective application window and the position states can be represented by a system of coordinates where the speech command input is based on absolute or relative coordinates. Advantageously a copy image of the graphical user interface of operation can be provided in front of or beside the actual user interface and a proxy pointer device is displayed on the copy image. In one embodiment, only the copy image comprises the discrete position states, and the speech input commands are only transferred to the copy image. Navigation of the proxy pointer device within the copy is transferred and converted into commands within the actual user interface. By the method of the '841 patent an operation event affected by a manipulation of the proxy pointer affects a corresponding event at the user interface. The '841 implementation implies the use of a grid-type overlay with the user moving the cursor on the grid.
U.S. Pat. No. 6,499,015, entitled “Voice Interaction Method for a Computer Graphical User Interface,” discloses an invention that enables a computer user to select a function represented via a graphical user interface by speaking command related to the function into audio processing circuitry. A voice recognition program interprets the spoken words to determine the function that is desired for execution. The user may use the cursor to identify an element on the graphical user interface display or speak the name of that element. The computer responds to the identification of the element by displaying a menu of the voice commands associated with that element. The '015 patent is focused on the execution of commands, not the positioning of a cursor.
U.S. Pat. No. 6,424,357, entitled “Voice Input System and Method of Using Same,” discloses a voice input system that has a microphone coupled to a computing device, with the computing device typically operating a computer software application. A user speaks voice commands into the microphone, with the computing device operating a voice command module that interprets the voice command and causes the graphical or non-graphical application to be commanded and controlled consistent with the use of a physical mouse. In one form of the invention, voice commands are overlayed on normal command points of the application program. Such voice commands are stored in a separate voice command repository, with an edit mode provided to create new voice command sequences and to edit existing voice command sequences. The '357 patent is focused on the execution of commands and fine positioning of the cursor.
U.S. Pat. No. 5,974,384, entitled “Window Control Apparatus and Method Having Function for Controlling Windows by Means of Voice-Input,” discloses a display unit that displays at least a window from among a plurality of windows. A window switching unit switches a currently active window between windows among the plurality of windows in accordance with a corresponding voice-input, the voice-input comprising relationship command indicating a relationship between the currently active window and a window to be activated from the windows among the plurality of windows. The relationship command indicates a relationship concerning an arrangement of the plurality of windows, the relationship comprising a relationship between the currently active window and a window to be activated from the windows among the plurality of windows. In many multi-display applications, including, for example, an aircraft cockpit environment, there is no particular concept of an “active window”. However, cursor positioning is required to be managed in an efficient manner.
U.S. Pat. No. 5,818,423, entitled “Voice Controlled Cursor Movement,” discloses a process in which an element (e.g., a location pointer in a windowed graphical user interface) shown on a computer display is moved toward a target position. A user's input is recognized which identifies a subregion of the display in which the target position is located. The element is moved discontinuously to a location within the identified subregion. Then a next user's input is recognized which identifies a sub-subregion of the subregion of the display in which the target position is located. The element is moved discontinuously to a location within the identified sub-subregion. In another aspect, an element is dragged between two locations on a computer display. User commands are accepted which indicate the two locations. Then the element is dragged between them. The user commands identify a sequence of discontinuous jumps for moving the element to the indicated location.
In many multi-display environments, in addition to being able to move the cursor in a coarse way to a region of interest, it is desirable to provide fine positioning of the cursor, and for both types of positioning to be provided in an efficient manner. This is critical, for example, in the aircraft cockpit environment.